Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New CPU-based Optimizer #1040

Merged
merged 18 commits into from
Mar 2, 2024
Merged

New CPU-based Optimizer #1040

merged 18 commits into from
Mar 2, 2024

Conversation

m4rs-mt
Copy link
Owner

@m4rs-mt m4rs-mt commented Aug 7, 2023

This PR adds a new meta optimizer to the Algorithms library which is designed for CPUs. It implements an optimization-performance and runtime-performance optimized version of the SGO algorithm:
Squid Game Optimizer (SGO): a novel metaheuristic algorithm [doi: 10.1038/s41598-023-32465-z].

The primary use case of this optimizer is to solve sophisticated optimization problems without the need to transform the input problem description into any intermediate state. Due to its efficiency it can be directly used in any .Net program to get excellent approximations of optimization results in near real time.

The implemented version does not implement the vanilla SGO algorithm from the paper. Instead, it uses modified update functions and specially tweaked position update logic using multiple buffers and tuned SGO-winner lists. These modifications of the original algorithm make this implementation significantly better in terms of optimization quality and runtime performance. Moreover, this version is fully parallelized and has the ability to use SIMD vector instructions to improve runtime performance.

This PR also adds a set of new optimizer test cases to ensure the convergence of the implemented approach.

Sample use case creating a scalar version of the optimizer while operating on floats to represent position vectors and evaluation results.

var random = new System.Random();
using var optimizer = MetaOptimizer.CreateScalar<
    float,
    float,
    RandomRanges.RandomRangeFloatProvider<XorShift64Star>>(
    random,
    1024 // Number of particles,
    2 // Number of dimensions);

optimizer.LowerBounds = new float[] { -1.0f, -1.0f };
optimizer.UpperBounds = new float[] { 1.0f, 1.0f };

var result = optimizer.Optimize(
    // Custom objective function here,
    // Custom break logic here,
    float.MaxValue // Best known solution);

@m4rs-mt m4rs-mt added the feature A new feature (or feature request) label Aug 7, 2023
@m4rs-mt m4rs-mt added this to the v2.0 milestone Aug 7, 2023
@m4rs-mt m4rs-mt force-pushed the cpu_optimizer branch 7 times, most recently from e36ff1a to 6fdfd42 Compare August 11, 2023 12:39
@m4rs-mt m4rs-mt force-pushed the cpu_optimizer branch 4 times, most recently from 675e1b0 to e5741b3 Compare August 22, 2023 07:35
@m4rs-mt m4rs-mt marked this pull request as ready for review August 22, 2023 09:34
m4rs-mt added a commit that referenced this pull request Aug 24, 2023
@m4rs-mt m4rs-mt force-pushed the cpu_optimizer branch 3 times, most recently from 40b8935 to 280711a Compare August 29, 2023 10:47
m4rs-mt added a commit that referenced this pull request Aug 30, 2023
m4rs-mt added a commit that referenced this pull request Aug 30, 2023
m4rs-mt added a commit that referenced this pull request Aug 30, 2023
@m4rs-mt m4rs-mt requested a review from MoFtZ September 4, 2023 21:49
@m4rs-mt m4rs-mt force-pushed the cpu_optimizer branch 3 times, most recently from 3e6ae59 to cc1d04d Compare September 5, 2023 10:28
@m4rs-mt m4rs-mt closed this Sep 5, 2023
@m4rs-mt m4rs-mt deleted the cpu_optimizer branch September 5, 2023 13:20
@m4rs-mt
Copy link
Owner Author

m4rs-mt commented Sep 5, 2023

This feature should actually live in a different repository as it is not directly ILGPU.Algorithms related.

Reopening this PR to include the proposed feature in the algorithms library.

…entations to adjust particle positions during optimization.
…aluation of objective functions on particles.
…ustomized and domain-specific objective functions.
…article positions in the SOG of the SGO algorithm in parallel.
…icle positions according to SGO in parallel.
@m4rs-mt m4rs-mt merged commit 18769cc into master Mar 2, 2024
63 checks passed
@m4rs-mt m4rs-mt deleted the cpu_optimizer branch March 2, 2024 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new feature (or feature request)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants